How Much Testing is Enough?
A forthcoming CPD event from the Safety and Reliability Society caught my eye. The topic is Reliability Demonstration Testing, a discipline concerned with establishing whether sufficient evidence exists to justify confidence in the performance of a system.
At first glance, that may sound like a niche reliability engineering subject. In reality, it raises a question that sits at the heart of many fire safety decisions.
How much testing is enough?
The fire sector is full of situations where we rely upon a sample of evidence and then draw conclusions about the performance of a wider system. We witness smoke control commissioning. We observe fire alarm cause-and-effect testing. We inspect a proportion of fire doors. We open up selected areas during compartmentation surveys. We review a sample of fire risk assessments as part of quality assurance processes.
In each case, there comes a point at which somebody decides that sufficient evidence has been gathered.
That judgement is rarely straightforward.
Consider a smoke control system serving a complex building. If a single test cycle is successful, we have demonstrated that the system can work. We have not necessarily demonstrated that it will work reliably under a range of conditions, after months of operation, or when called upon during a real fire event.
Similarly, if a compartmentation survey identifies defects in several locations, what level of confidence should we have that other concealed areas are satisfactory? If a cause-and-effect test demonstrates the correct operation of a sample of alarm interfaces, how much confidence should we place in the wider matrix?
These are not purely technical questions. They are questions about evidence, uncertainty and professional judgement.
Reliability engineers have spent decades developing methods to determine whether sufficient testing has been undertaken to justify a particular level of confidence. Fire safety professionals often encounter the same challenge, albeit in different forms. The objective is not simply to demonstrate that something works once. The objective is to determine whether enough evidence exists to support the conclusions being drawn.
That does not mean every fire safety project requires formal reliability modelling. It does suggest that we should be careful whenever the justification for adequacy becomes little more than "we tested it and it worked".
A more useful question may be: what level of confidence does this testing actually provide?
For systems relied upon to support means of escape, smoke control, firefighting operations or business continuity, the answer matters. The consequences of getting it wrong can be significant.
As the fire safety profession continues to place greater emphasis on evidence, assurance and competence, there may be value in borrowing more thinking from adjacent disciplines. Reliability engineering has much to say about uncertainty, confidence and the limits of testing. Those concepts are every bit as relevant to fire safety as they are to aerospace, defence or manufacturing.
Ultimately, the challenge is not demonstrating that a system worked during commissioning. The challenge is establishing sufficient confidence that it will perform when the building experiences the one fire that truly tests the design.
Disclaimer: Pyrology Insights are intended to encourage professional discussion and reflection. They do not constitute project-specific advice and should not be relied upon in place of appropriate professional judgement.